{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 离线批量推理\n",
    "\n",
    "在进行模型评估或其他任务时，通常需要对大量数据进行预测。然而，模型推理过程往往耗时较长，通过循环串行执行会增加整体时间成本，而并行执行则需要额外的开发工作。\n",
    "\n",
    "SDK 提供了多种解决方案来应对这一场景，其中包括：\n",
    "\n",
    "- [本地并行推理](https://github.com/baidubce/bce-qianfan-sdk/blob/main/cookbook/batch_prediction.ipynb)：利用 SDK 内置的批量推理功能，在本地通过并行调用模型接口实现高效的批量预测。\n",
    "- [数据集评估](https://github.com/baidubce/bce-qianfan-sdk/blob/main/cookbook/dataset/batch_inference_using_dataset.ipynb)：利用 SDK 的 Dataset 模块，调用平台提供的数据集评估功能，以便快速而有效地完成任务。\n",
    "- [离线批量推理](https://github.com/baidubce/bce-qianfan-sdk/blob/main/cookbook/offline_batch_inference.ipynb)：对于时间要求不那么严格的场景，可以考虑利用平台提供的离线批量预测能力，以降低实时推理的负载压力。\n",
    "\n",
    "本文将介绍第三种解决方案，即离线批量推理。\n",
    "\n",
    "注意：本文的方法需要 SDK 版本 >= 0.3.4"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "使用前需要先将数据上传至 BOS 上，数据格式如下：\n",
    "\n",
    "```\n",
    "{\"id\": \"1\", \"query\": \"地球的自转周期是多久？\"}\n",
    "{\"id\": \"2\", \"query\": \"太阳系中最大的行星是哪颗？\"}\n",
    "{\"id\": \"3\", \"query\": \"月亮是围绕地球还是围绕太阳运转的？\"}\n",
    "{\"id\": \"4\", \"query\": \"水的化学式是什么？\"}\n",
    "{\"id\": \"5\", \"query\": \"世界上最高的山是哪座？\"}\n",
    "```\n",
    "\n",
    "之后可以通过如下方式创建一个离线批量推理任务"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "from qianfan.resources.console.data import Data\n",
    "\n",
    "bos_name=\"your_bucket_name\"\n",
    "\n",
    "task = Data.create_offline_batch_inference_task(\n",
    "    name=\"random_task_name\",  # 任务名称\n",
    "    descrption=\"task_description\",  # 任务描述，可选\n",
    "    # 推理模型的链接\n",
    "    endpoint=\"https://aip.baidubce.com/rpc/2.0/ai_custom/v1/wenxinworkshop/chat/completions\",\n",
    "    inference_params= {   # 模型推理参数\n",
    "        \"temperature\": 0.5,\n",
    "    },\n",
    "    input_bos_uri=f\"bos:/{bos_name}/inference-input/\",  # 输入文件在 BOS 的路径\n",
    "    output_bos_uri=f\"bos:/{bos_name}/inference-output/\"  # 输出文件在 BOS 的路径\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "并通过如下方式获取任务的 id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "task_id = task['result']['taskId']\n",
    "print(task_id)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "根据 id 就可以获取任务的详情，得到任务的进度，等待任务完成，就可以从输出的 BOS 地址获取推理完成后的结果。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Done\n"
     ]
    }
   ],
   "source": [
    "task_info = Data.get_offline_batch_inference_task(task_id=task_id)\n",
    "\n",
    "status = task_info['result']['runStatus']\n",
    "print(status)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "py311",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
